[WIP] OCPBUGS-56269: Remove registry cache placeholder on scrape failure #1025

PratikMahajan · 2025-05-15T21:05:38Z

replace the placeholders with None
so no more 0.0.0

and

remove the placeholder from the cache of we fail to get any release
metadata

openshift-ci-robot · 2025-05-15T21:05:46Z

@PratikMahajan: This pull request references Jira Issue OCPBUGS-56269, which is valid. The bug has been moved to the POST state.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.20.0) matches configured target version for branch (4.20.0)
bug is in the state New, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @jiajliu

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

openshift-ci · 2025-05-15T21:05:58Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: PratikMahajan

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [PratikMahajan]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Current production Cincinnati occasionally confuses arch-specific shards of multi-arch images as single-arch releases Possibly triggered by a race or other buggy handling of Quay 502s while scraping releases. fix: check if the metadata is multi for layered and registry and ignore those single arch shards Co-authored-by: wking <wking@tremily.us>

This reverts commit 9713747.

replace the placeholders with `None` so no more 0.0.0 and remove the placeholder from the cache of we fail to get any release metadata Co-authored-by: wking <wking@tremily.us>

openshift-ci-robot · 2025-05-15T21:38:42Z

@PratikMahajan: This pull request references Jira Issue OCPBUGS-56269, which is valid.

3 validation(s) were run on this bug

bug is open, matching expected state (open)
bug target version (4.20.0) matches configured target version for branch (4.20.0)
bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)

Requesting review from QA contact:
/cc @jiajliu

Details

In response to this:

replace the placeholders with None
so no more 0.0.0

and

remove the placeholder from the cache of we fail to get any release
metadata

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

wking · 2025-05-15T22:07:33Z

cincinnati/src/plugins/internal/graph_builder/release_scrape_dockerv2/registry/mod.rs

            Err(e) => {
-                debug!(
+                return Err(format_err!(
                    "[{}] Could not assemble metadata from layer ({}): {}",


Hey look, there's some CI coverage in cargo-test:

cincinnati: plugins::internal::graph_builder::release_scrape_dockerv2::plugin::network_tests::scrape_public_with_first_empty_tag_must_succeed expand_less 1s {Error: failed to fetch all release metadata from registry.ci.openshift.org/cincinnati-ci-public/cincinnati-test-emptyfirsttag-public-manual test failure with exit code 101 Error: failed to fetch all release metadata from registry.ci.openshift.org/cincinnati-ci-public/cincinnati-test-emptyfirsttag-public-manual Caused by: [0.0.0] Could not assemble metadata from layer (sha256:4ca545ee6d5db5c1170386eeb39b2ffe3bd46e5d4a73a9acbebc805f19607eb3): 'release-manifests/release-metadata' not found} cincinnati: plugins::internal::graph_builder::release_scrape_dockerv2::plugin::network_tests::scrape_public_multiarch_manual_succeeds expand_less 1s {Error: failed to fetch all release metadata from registry.ci.openshift.org/cincinnati-ci-public/cincinnati-test-public-multiarch-manual test failure with exit code 101 Error: failed to fetch all release metadata from registry.ci.openshift.org/cincinnati-ci-public/cincinnati-test-public-multiarch-manual Caused by: [0.0.3-amd64] Could not assemble metadata from layer (sha256:4ca545ee6d5db5c1170386eeb39b2ffe3bd46e5d4a73a9acbebc805f19607eb3): 'release-manifests/release-metadata' not found}

I'm not clear yet on whether that's turning up fixtures/tests we should adjust, or pointing out issues in our new code.

ah, seems like that's "we didn't find find release-manifests/release-metadata in the top layer", and we need to adjust our error handling here to allow it to continue further in the for layer_digest in layer_digests loop, in case we can find the metadata file in a deeper layer. I'm not sure how to express that in Rust 😅 I think we might need to adjust assemble_metadata to return an Option, so it can say "I was successfully able to process this blob, but I didn't find metadata_filename inside it"?

openshift-ci · 2025-08-20T16:59:51Z

@PratikMahajan: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e	`ed9c72f`	link	true	`/test e2e`
ci/prow/cargo-test	`ed9c72f`	link	true	`/test cargo-test`
ci/prow/olm-e2e	`ed9c72f`	link	true	`/test olm-e2e`
ci/prow/verify-openapi	`ed9c72f`	link	true	`/test verify-openapi`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-bot · 2025-11-19T01:01:19Z

Issues go stale after 90d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle stale

coderabbitai · 2025-11-19T01:01:58Z

Important

Review skipped

Auto reviews are limited based on label configuration.

🚫 Review skipped — only excluded labels are configured. (1)

do-not-merge/work-in-progress

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

openshift-bot · 2025-12-19T08:30:42Z

Stale issues rot after 30d of inactivity.

Mark the issue as fresh by commenting /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.
Exclude this issue from closing by commenting /lifecycle frozen.

If this issue is safe to close now please do so with /close.

/lifecycle rotten
/remove-lifecycle stale

openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 15, 2025

openshift-ci-robot added jira/severity-moderate Referenced Jira bug's severity is moderate for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. labels May 15, 2025

openshift-ci-robot added the jira/valid-bug Indicates that a referenced Jira bug is valid for the branch this PR is targeting. label May 15, 2025

openshift-ci bot requested a review from jiajliu May 15, 2025 21:05

openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 15, 2025

PratikMahajan force-pushed the remove-cache-if-no-rel-mtdta branch from d2dddf1 to b806084 Compare May 15, 2025 21:29

PratikMahajan and others added 3 commits May 15, 2025 21:31

Revert "do not panic on sha-mismatch for multi-arch images"

fe23e23

This reverts commit 9713747.

remove registry-cache placeholders on release scrape failures

ed9c72f

replace the placeholders with `None` so no more 0.0.0 and remove the placeholder from the cache of we fail to get any release metadata Co-authored-by: wking <wking@tremily.us>

PratikMahajan force-pushed the remove-cache-if-no-rel-mtdta branch from b806084 to ed9c72f Compare May 15, 2025 21:38

wking reviewed May 15, 2025

View reviewed changes

openshift-ci bot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 19, 2025

openshift-ci bot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 19, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[WIP] OCPBUGS-56269: Remove registry cache placeholder on scrape failure #1025

[WIP] OCPBUGS-56269: Remove registry cache placeholder on scrape failure #1025

Uh oh!

PratikMahajan commented May 15, 2025 •

edited

Loading

Uh oh!

openshift-ci-robot commented May 15, 2025

Uh oh!

openshift-ci bot commented May 15, 2025

Uh oh!

openshift-ci-robot commented May 15, 2025

Uh oh!

wking May 15, 2025

Uh oh!

wking May 15, 2025 •

edited

Loading

Uh oh!

openshift-ci bot commented Aug 20, 2025

Uh oh!

openshift-bot commented Nov 19, 2025

Uh oh!

coderabbitai bot commented Nov 19, 2025 •

edited

Loading

Review skipped

Uh oh!

openshift-bot commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[WIP] OCPBUGS-56269: Remove registry cache placeholder on scrape failure #1025

Are you sure you want to change the base?

[WIP] OCPBUGS-56269: Remove registry cache placeholder on scrape failure #1025

Uh oh!

Conversation

PratikMahajan commented May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-ci-robot commented May 15, 2025

Uh oh!

openshift-ci bot commented May 15, 2025

Uh oh!

openshift-ci-robot commented May 15, 2025

Uh oh!

wking May 15, 2025

Choose a reason for hiding this comment

Uh oh!

wking May 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Aug 20, 2025

Uh oh!

openshift-bot commented Nov 19, 2025

Uh oh!

coderabbitai bot commented Nov 19, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

openshift-bot commented Dec 19, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

PratikMahajan commented May 15, 2025 •

edited

Loading

wking May 15, 2025 •

edited

Loading

coderabbitai bot commented Nov 19, 2025 •

edited

Loading